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Abstract 

There is a remarkable connection between the maximum clique number and the Lagrangian of 
a graph given by T. S. Motzkin and E.G. Straus in 1965. This connection and its extensions were 
successfully employed in optimization to provide heuristics for the maximum clique number in graphs. 
It is useful in practice if similar results hold for hypergraphs. In this paper, we explore evidences that 
the Lagrangian of a 3-uniform hypergraph is related to the order of its maximum cliques when the 
number of edges of the hypergraph is in certain range. In particular, we present some results about 
a conjecture introduced by Y. Peng and C. Zhao (2012) and describe a combinatorial algorithm that 
can be used to check the validity of the conjecture. 

1 Introduction 

In 1941, Turan [22] provided an answer to the following question: What is the maximum number of 
edges in a graph with n vertices not containing a complete subgraph of order I, for a given 1? This is 
the well-known Turan theorem. In 1965, another classical paper by Motzkin and Straus [5] provided a 
new proof of Turan's theorem based on a continuous characterization of the clique number of a graph 
using the Lagrangian of a graph. This new proof aroused interests in the study of Lagrangians of 
hypergraphs. Furthermore, the Motzkin-Straus result and its extension were successfully employed in 
optimization to provide heuristics for the maximum clique problem, and the Motzkin-Straus theorem 
has been also generalized to vertex-weighted graphs [6J and edge-weighted graphs with applications to 
pattern recognition in image analysis (see [T], [2J, [5], [5J, [5], [TU], [II])- It is useful in practice if similar 
results hold for hypergraphs. In this paper, we provide evidences that the Lagrangian of an r-uniform 
hypergraph is related to the order of its maximum cliques under some conditions. We first state a few 
definitions. 

For a set V and a positive integer r we denote by the family of all r-subsets of V. An r-uniform 
graph or r-graph G consists of a set V(G) of vertices and a set E(G) C V(G)^ of edges. An edge 
e = {a±, ci2, ■ ■ ■ , a r } will be simply denoted by a\ai . . . a r . An r-graph H is a subgraph of an r-graph 
G, denoted by H C G if V(H) C V(G) and E(H) C E(G). Let N be the set of all positive integers. 
For any integer n G N, denote the set {1, 2,3, ... ,n} by [n]. Let denote the complete r-graph on t 
vertices, that is the r-graph on t vertices containing all possible edges. A complete r-graph on t vertices 
is also called a clique with order t. We also let [n]^ represent the complete r-graph on the vertex set 
[n]. When r = 2, an r-graph is a simple graph. When r > 3, an r-graph is often called a hypergraph. 
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Definition 1.1 Let G be an r-graph with vertex set [n] and edge set E{G). Let S = {x = (xi, X2, • ■ • , x n ) 
G R n : Y17=i Xi = 1) > for i = 1, 2, . . . , n}. For x — (xi,X2, ■ • ■ , x n ) G S, define 

A(G, ^ ^ Xi-^ Xi 2 ■ ■ • Xi T . 

i 1 i 2 ---i r eE(G) 

The Lagrangian of G, denoted by A(G) ; is defined as A(G) = max{A(G, x) : x G S}. A vector y G S is 
called an optimal weighting for G if X(G,y) — X(G). 

The following fact is easily implied by the definition of the Lagrangian. 

Fact 1.1 Let G\, G2 be r-uniform graphs and G\ C G2. Then A(Gi) < A(G2). 

In [5] , Motzkin and Straus proved that the Lagrangian of a 2-graph is determined by the order of its 
maximum clique. 

Theorem 1.2 (Motzkin and Straus If G is a 2-graph in which a largest clique has order I then 
A(G) = A(X ; (2) )=A([;p)) = I(l-I). 

The obvious generalization of Motzkin and Straus' result to hypergraphs is false because there are 
many examples of hypergraphs that do not achieve their Lagrangian on any proper subhypergraph. 
An attempt to generalize the Motzkin-Straus theorem to hypergraphs is due to Sos and Straus [T7] , 
Recently, in [Tl] and [TS] Rota Bulo and Pelillo generalized the Motzkin and Straus' result to r-graphs 
in some way using a continuous characterization of maximal cliques with applications in image analysis. 
Although, the obvious generalization of Motzkin and Straus' result to hypergraphs is false, we attempt 
to explore the relationship between the Lagrangian of a hypergraph and the order of its maximum cliques 
for hypergraphs when the number of edges is in certain range. In |12j . the following two conjectures are 
proposed. 

Conjecture 1.3 (Peng-Zhao f!2f) Let I andm be positive integers satisfying ( ~ 1 ) <m< ( i ~ 1 ) + ('~ 2 ). 
Let G be an r-graph with m edges containing a clique of order I — 1. Then A(G) = X([l — l]( r )). 

Conjecture 1.4 (Peng-Zhao U2]j ) Let G be an r-graph with m edges and without containing a clique 
of order I - 1, where < m < f; 1 ) + ( l r Z\) ■ Then A(G) < X([l - 

The upper bound ( l ^ r 1 ) + ( _i) m Conjecture 11.31 is the best possible. For example, if m = + 
(r-i) + 1 then A(G r!m ) > A([/ — l] 1 -*''), where C r , m is the r-graph on the vertex set [/] and with the edge set 
[Z-l]( r )u{ii---i r _iZ,ii---i r _i e [2-2]( , - 1 )}U{l---(r-2)(Z-l)Z} (note that, take x = (x lt ...,xi) 6 S, 
where x\ = X2 = ■ ■ ■ = xi- 2 = -rhj and xi-% = xi = 2 ^_^ , then A(£7 rjm ) > A(C r , m , x) > A([Z - l]^ r ^).) 

In the course of estimating Turan densities of hypergraphs by applying the Lagrangians of related 
hypergraphs, Frankl and Fiiredi [1] asked the following question: Given r > 3 and m € N how large 
can the Lagrangian of an r-graph with m edges be? In order to state their conjecture on this problem 
we require the following definition. For distinct A, B € we say that A is less than B in the colex 
ordering if max(AAB) G B, where AAB = (A \ B) U (B \ A) is the symmetric difference of A and B. 
For example we have 246 < 156 in since mox({2,4,6}A{l,5,6}) G {1,5,6}. Let C r . m denote the 
r-graph with m edges formed by taking the first m elements in the colex ordering of W r > . 

The following conjecture of Frankl and Fiiredi (if it is true) proposes a solution to the above question. 

Conjecture 1.5 (Frankl and Fiiredi J4H) The r-graph with m edges formed by taking the first m sets 
in the colex ordering of has the largest Lagrangian of all r-graphs with m edges. In particular, the 
r-graph with ( ) edges and the largest Lagrangian is [l]( r \ 
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This conjecture is true when r = 2 by Theorem 11.21 For the case r = 3, Talbot in [18] proved the 
following. 

Theorem 1.6 (Talbot J18f) Let m and I be integers satisfying 

Then Conjecture \1.5\ is true for r = 3 and this value of m. 

In [19] and [20], Theorem [T76] is improved as follows. 
Theorem 1.7 (Tang, Peng, Zhang, and Zhao) Let m and I be integers satisfying 

TTien Conjecture Q73] is £rwe /or r = 3 and t/ws vahie of m. 

Remark 1.8 TVie truth of Frankl and Fiiredi's conjecture is not known in general for r > 4. Even in 
the case r — 3, the case when (3) — 6 < to < g 1 ) + (' — (J — 4) is still open in this conjecture. In 
J7j] and \20]j , Frankl and Fiiredi's conjecture for small values m is verified when r — 3. 

The following result is given in [TB] . 

Lemma 1.9 (Talbot J18f) For any integers m,l, and r satisfying (~ ) < to < ( + ( r Zi)j we have 
A(C r , m ) = X([l - 

Remark 1.10 Based on Lemma \l.9l we can see that if both Conjectures \1.3\ and \ l-4\ are true, then 
Conjecture \1.5\ is true for this range of to. It is easy to see that if (~ J < to < ( _ ) + ( r Zi)j then 
Conjecture \1.5\ implies Conjecture \1.S\ 

In [12], it has been shown that Conjecture 11.31 holds when r = 3. 

Theorem 1.11 (Peng-Zhao U2tf ) Let to and I be positive integers satisfying ('g 1 ) < to < ('g 1 ) + C^ 2 )- 
Lei G be a 3 -graph with to edges and contain a clique of order I — 1. TTien A(G) = X([l — l]^)- 

In [11], it has been shown that Conjecture II .41 holds when r = 3 with some conditions. 

Theorem 1.12 (Peng- Tang- Zhao fllJj) Let m and I be positive integers satisfying (' g 1 ) < to < ('g 1 ) + 
~ 2 ) — (I — 2). Lei G be a 3-graph with to edges and without containing a clique of order I — 1. Then 
A(G)<A([Z-lp)). 

Further evidences for Conjectures 11.31 and 11.41 are provided in [TT] and [31] when the number of 
vertices of the r-graph is restricted to be a fixed number I. 

In this paper, we provide more evidence for Conjecture 11.41 when r = 3. In particular, to confirm 
Coniecture ll.4l when r = 3, we prove that we only need to consider 3-graphs with fixed number I vertices, 
and thus we may describe a combinatorial algorithm of verifying Conjecture 11.41 for given I vertices in 
Section [3] We point out that a preliminary related partial results was given in [13] . Let us state some 
useful results in the following section. 
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2 Preliminary Results 

For an r-graph G = (V, E) we denote the (r — l)-neighborhood of a vertex i € V by E t = {A £ y( r_1 J : 
A U {i} 6 i?}. Similarly, we will denote the (r — 2)-neighborhood of a pair of vertices i,j £ V by 
Eu = {B £ V( r - 2 1 : B U {i,j} £ E}. We denote the complement of E t by Ef = V^ 1 ) : Au{i} £ 
V^\E}. Also, we will denote the complement of E i6 by Ef 3 = {B £ F (r_2) : 5 U {i, j} £ F^VE}. 
Denote 

= Ei n E°. 

We will impose one additional condition on any optimal weighting x — (x\,X2, ■ ■ ■ ,x n ) for an r-graph 
G: 

\{i : Xi > 0}| is minimal, i.e. if y £ S satisfying \{i : y t > 0}| < \{i : Xi > 0}|, then X(G,y) < X(G). (1) 

When the theory of Lagrange multipliers is applied to find the optimum of A(G), subject to Xh=i x < = 
1, notice that X(Ei,x) corresponds to the partial derivative of X(G,x) with respect to a;,. The following 
lemma gives some necessary condition of an optimal weighting of A(G). 

Lemma 2.1 (Frankl and Rodl Let G = (V,E) be an r-graph on the vertex set [n] and x = 
(xi,X2, ■ ■ ■ ,x n ) be an optimal weighting for G with k (< n) non-zero weights Xi, X2, . . . , x% satisfy- 
ing condition (QJ). Then for every {i,j} £ [k]^ , (a) X(Ei,x) — X(Ej,x) — rX(G), (b) there is an edge 
in E containing both i and j . 

Lemma 2.2 Suppose G is an r-graph on the vertex set [n] with edge set E. Let 1 < i < j ' < n. If 
Ej\A — 0, then there exists an optimal weighting y — (j/i, 1/2, ■ • ■ , Vn) of A(G) such that y% > yj. 

Proof. Let x = (x\,X2, ■ ■ ■ ,x n ) be an optimal weighting for G. If Xi < Xj, then let yk = Xk for 
kj^i,j,Vi = xj and yj = x { . Then y = (y 1; y 2 , . . . , y n ) is a weighting for G with y { > yj and 

X{G,y) - A(G, x) = (xj - Xi)(X(E Aj ,x) - X(E ]V ,x)) > 0. 

So y is an optimal weighting satisfying the condition. M 
We call two vertices i,j of an r-graphG = (V,E) equivalent if for all / £ (^-f 7 ^)' / G if an d 
onlyif/G^. 

Lemma 2.3 (c.f. Joy) Suppose G is an r-graph on the vertex set [n]. If vertices i\, i 2 , it are 
pairwisely equivalent, then there exists an optimal weighting y = (yi, y 2 , ■ ■ ■ , y n ) of A(G) such that y^ — 
Vi2=--- = Vit ■ 

We say that an r-graph G = (V, E) on the vertex set [n] is left compressed if Ej\s = for any 
1 < i < j ^ n - In other words, for any i < j, if k\k 2 ■ ■ - k r -\ £ Ej, where k\, k,2, ■ ■ ■ , fc r — 1 7^ i, 
then k\k 2 ■ ■ ■ k r —i £ Ei.Equivalently, an r-graph G = (V,E) is left compressed if jij 2 ■ ■ ■ j r £ E implies 
i\ i 2 ■ ■ ■ i r £ E provided i p < j p for every p, 1 < p < r. We will define right compressed similarly later. 

Remark 2.4 (a) In Lemma ] 2. 11 part(a) implies that XjX(Eij,x) + X(E i \j,x) = XiX(Eij,x) + X(Ej\ i ,x). 
In particular, if G is left compressed, then 

(xi - Xj)X(Eij,x) = X(Ei\j,x) (2) 

for any i,j satisfying 1 < i < j < k since Ej\i = 0. 

(b) By (0), if G is left- compressed, then an optimal weighting x = {x\, x 2 , ... , x n ) for G must satisfy 

X\ > X2 > . . . > x n > 0. (3) 
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(k-5)(k-1)k (k-4)(k-2)k (k-3)(k-2)(k-1) 

0<^5<^)k~~M)(k-2)(k-1 ) 

(k-7^k-1)k (k-6)(k-2)k (k-5)(k-3)k (k-5)(k-2)(k-1) (k-4)(k-3)(k-1) 



(k-8)(k-1)k (k-7)(k-2)k (k-6)(k-3)k (k-6J(k-2)(k-1) (k-5)(k-4)k (k-5)(k-3)(k-1) (k-4)(k-3)(k-2) 
I I I I II I 

I I I I II I 



Figure 1: Hessian diagram of triples on k vertices 



3 Evidences for Conjecture 11.4 



In this section, we describe a procedure that can be used to confirm Conjecture II .41 when r = 3 and I is 
small. Denote A£~ « = max{A(G) : G is an r - graph with m edges and does not contain a clique of size 

I}- ^ ! """ 

We need a reduction lemma to simplify our procedure for verification of Conjecture [L4] 



Lemma 3.1 Let m and I be positive integers satisfying 

left compressed 3-graph G on the vertex set [I] with m edges such that A(G) = A 3 



(' a 1 ) < - < (Y) 



n-2\ 



Then there exists a 



(m,l—l) ' 



Remark 3.2 Lemma \3.1\ is used in proving some partial results for confirming Conjecture \l-4\ i n JHI 
and \2iy . A similar lemma was given in U3f , however that proof was flawed. 



To verify Lemma l3.ll we define a partial order Hessian diagram K, a 2-graph on vertices of all triples. 
A triple ii«2*3 is called an ancestor of a triple j\ j 2 j'3 if i\ > j\, i 2 > 32, *3 > 33, and«i+t2+i3 > J1+J2+J3. 
In this case, the triple J1J2J3 is called a descendant of iii 2 i3- We say that ii?2*3 has higher hierarchy 
than J1J2J3 if 21*2*3 is an ancestor of jxjijz- A triple 1112*3 is called a directed ancestor of a triple jij^ja 
if h > ji, h > 32, h > 33, and i x + i 2 + i 3 = ji + ji + ja + 1- In this case, the triple jij 2 j3 is called 
a directed descendant of «i«2*3, and vertices iii 2 i 3 and jij 2 j 3 are adjacent in the corresponding Hessian 
diagram. Note that hierarchy is a partial ordering. Figure 1 shows part of the hierarchy relationship 
of triples in Note that a 3-graph G is left-compressed if and only if for any edge in G, all its 

descendants (in Hessian diagram K) should be in G as well. 



Proof of Lemma 13.11 Let G be a 3-graph on the vertex set [n] with m edges without containing a 
clique of order I — 1 such that A(G) = A 3 ~ l _ 1 y We call such a 3-graph G an extremal 3-graph for m 
and I — 1. Let x — {x\, x 2 , . . . , x n ) be an optimal weighting of G. We can assume that X{ > Xj when 
i < j since otherwise we can just relabel the vertices of G and obtain another extremal 3-graph for m 
and / — 1 with an optimal weighting x — (xx, x 2 , . . . , x n ) satisfying Xi > Xj when i < j. Next we obtain 
a new 3-graph H from G by performing the following: 



1. If (I — 3)(Z — 2)(l — 1) 6 E(G), then there is at least one triple in 
(I - 3)(Z - 2)(l - 1) by this triple; 



1]< 3 ) \ E(G), we replace 



2. If an edge in G has a descendant other than (I — 3)(Z - 
this edge by a descendant other than (I — 3)(Z — 2)(Z 
until there is no such an edge. 



2)(Z — 1) that is not in E(G), then replace 
- 1) with the lowest hierarchy. Repeat this 



5 



Then H satisfies the following properties: 



1. The number of edges in H is the same as the number of edges in G. 

2. A(G) = X(G, x) < X{H,x) < X(H). 

3. (Z-3)(Z-2)(Z-1) $E(H). 

4. For any edge in E(H), all its descendants other than (Z — 3)(Z — 2)(Z — 1) will be in E(H). 

If H is not left-compressed, then there is an ancestor uvw of (Z— 3)(Z — 2)(Z — 1) such that wuw G E(H). 
We claim that www; must be (Z — 3)(Z — 2)1. If uvw is not (/ — 3)(/ — 2)1, then since all descendants other 
than (Z — 3)(Z — 2)(Z — 1) of uvw will be in E(H), then all descendants of (/ — 3)(Z — 1)/ (other than 
(Z - 3)(Z - 2){l - 1)) or all descendants of (/ - 3)(Z - 2)(Z + 1) (other than (Z - 3)(Z - 2)(Z - 1)) will 
be in E{H). So all triples in [Z - 1] (3) \ {(I - 3)(Z - 2)(Z - 1)}, all triples in the form of ijl (where 
ij G [Z — 2]( 2 )), and all triples in the form of ij(l + 1) (where ij G [Z — 2]( 2 )) or all triples in the form of 
i(l - 1)1, 1 < % < I - 3 will be in E(H), then 

which is a contradiction. So uvw must be (I — 3)(Z — 2)Z. Since m < ( g 1 ) + ( 2 2 ) and all the descendants 
other than (Z — 3)(Z — 2)(Z — 1) of an edge in H will be an edge in H , then there are two possibilities. 

• Case 1. E(H) = {[I - 1]( 3 ) \ {{I - l)(l - 2)(Z - 3)}) U {ijl,ij G [Z - 2]( 2 '} U {12(1 + 1)}. 

• Case 2. E(H) = ([I - 1}^ \ {{I - l)(l - 2){l - 3)}) U {ijl,ij G [I - 2]< 2 )}. 

Let y = (j/i, y2, ■ ■ ■ , y n ) be an optimal weighting of H, where n = I + 1 or n = I. We claim that if 
Case 1 happens, then yi + i — 0. Notice that = for each 1 < i < I, by Lemma [2.2[ we can 

assume that yi > yi + \ for each 1 < i < I. If yi + i > 0, then each t/, > 0. This contradicts to = 

by Lemma \2. 11 So we may just consider Case 2. 

Since I — 1, Z — 2 and Z — 3 are pairwisely equivalent in H (Case 2), then by Lemma 12. 3[ we can 
assume that = y^ 2 = Vis- K = 2/i-2 = yi-3 = 0, then clearly X(H) = X(H, y) < A([Z - 1] (3) ), 
therefore, A(G) < A([Z — l]^ 3 -*) which confirms Conjecture 11.41 So assume that yi-\ = yi-i = yi-z > 0. 
Let F be the 3-graph with edge set 

E(F) = ({l-l}^U{i3l,ije{l-2}W}. 

Then 

A(G) < X(H) = X(H,y) < X(F,y) < X(F) = X([l - 1}^) 

which conhrms Conjecture II .41 So we can assume that H is left-compressed. Clearly H does not contain 
[Z — 1]( 3 ). Since H is left-compressed, then H does not contain a clique of order I — 1. Therefore we get 
a left-compressed extremal 3-graph H for m and Z — 1. Hence we can assume that G is left -compressed. 

Next show that we can assume G is on I vertices. Let k be the number of positive coordinates in x 
(optimal weighting for G). If k < I - 1, then A(G) < A([Z - 1]< 3 ) \ {(I - 1)(Z - 2)(Z - 3)}) < A([Z - 1]( 3 )) 
which confirms Conjecture 11.41 So we may assume that k > I. We will use Lemma 13.31 below. The 
proof of Lemma 13.31 is similar to a proof of a result in [18 . However, the result in |18j cannot be applied 
directly here. For completeness, the proof of Lemma 13.31 will be given in the appendix. 

Lemma 3.3 Let m and I be positive integers satisfying < rn < ('^j 1 ) + C^ 2 )- Let G be a left- 

compressed 3-graph on the vertex set [k] with m edges and without containing a clique of order I — 1 such 
that A(G) = A 3 m j_]v Let x be an optimal weighting for G with k positive weights. Then Confecture \l.J\ 
holds or 

|[fc - 1] (3) \E\ <k-2. 
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Now assume that Lemma [3751 holds and continue the proof of Lemma [3~T1 Since G is left compressed, 
then l(k — \)k G E and |[jfe - 2]< 2 ) f\E k \ > 1. 

If A; > Z + 1, then applying Lemma |3.3[ we have 

m = \E\ 

■ 

> 

which contradicts to the assumption that m < ('g 1 ) + C^ 2 )- Recall that k > I, so we have k = I. Since 
x has only I positive weights, we can assume that G is on I vertices. This proves Lemma 13. II H 

Theorem 3.4 To verify Conjecture \l-4\ for r — 3 and any given I, it is sufficient to verify X(G) < 
X([l — 1]^) for all left-compressed 3-graphs G on the vertex set [I] with m — ('g 1 ) + ( 2 ) edges and 
without containing the clique [I — 1]^ . 

Proof: Since A?~ does not decrease as m increases, it is sufhcient to verify Conjecture 11.41 for 
m = ('j 1 ) + Cj 2 )- By Lemma IBTTl it is sufficient to verify A(G) < A([Z - 1] (3) ) for all left compressed 
3-graphs G on I vertices with m = ( g 1 ) + ('j 2 ) edges and without containing the clique [/ — 1]( 3 ). 

Now we describe an Algorithm to produce all left compressed 3-graphs G on the vertex set [I] with 
m = ('g 1 ) + ('~ 2 ) edges and without containing the clique [I — 1]( 3 '. Notice that, for a 3-graph G on I 
vertices with m = (' g 1 ) + ( ^ 2 ) edges and without containing a clique of order I — 1, we may write [Z]^ 3 ) 
as 

[Z]< 3 > = [i-l]( 3 >UGiUG 2 

where G x = {ijl, where ij € [/ -2]< 2 )} and G 2 = {1(1 - 1)1, 2(1 - 1)1, 3(1 - 1)1, ■ ■ ■ , (I - 2)(i - 1)1}. 

Under these assumptions, G can be obtained from [l]^ by deletion of a subgraph iJ with I — 2 edges 
since I - 2 = ( 3 ) - [(' g 1 ) + (' 2 2 )]. These Z - 2 edges of H consist of edges from [I - lp\ Gi (if any), or 
G2. Specifically, 3-graph G takes the form of 

G=([l- 1] (3) - Si) U (Gi - S 2 ) U {1(1 - 1)/, 2(1 - 1)1, 3(1 - 1)1, 1)1} 

for some edge set E\ from [I — l]*- 3 -*, some edge set E2 from Gi, and some i where 1 < i < I — 3. Observe 
that E q \j = for 1 < q < j < i. By ©, x\ = x 2 = ■ ■ ■ = x t . 

An r-graph H = (V,E) on the vertex set [I] is right-compressed if 1112*3 S E implies jijzjs G E 
whenever j\ > i\, j'2 > i%, and j'3 > 13. Note that H is right-compressed if and only if the complement 
of H is left-compressed. To generate all possible left compressed 3-graphs on the vertex set [I] with 
m = ( g 1 ) + ( ~ 2 ) edges and without containing the clique [I — 1]^ 3 \ we can generate all right compressed 
connected 3-subgraphs H rooted at (I — 2) (I — 1)1 with I — 2 edges and containing (I — 3)(l — 2)(l — 1) 
and then take the complement of each H. To do so, we use Algorithm 13.51 in the following procedure 
that is based on Figure 1 (replacing k by I). 

Algorithm 3.5 List all left compressed 3-graphs on the vertex set [I] with m = ( g 1 ) + ( ~ 2 ) edges and 
without containing the clique [I — 1]( 3 ). 

Input: I > 7 and a Hessian graph with vertex set [l]^ (replace k by I in Figure 1). 

Output: All right compressed connected 3-subgraphs H rooted at (I — 2) (I — 1)1 with I — 2 edges and 
containing (I — 3)(l — 2)(l — 1), thus produce all possible left compressed 3-graph G — [1]^ — H on the 
vertex set [I] with m — ('g 1 ) + ( l ^ 2 ) edges and without containing the clique [I — l]*- 3 '. 

Initialization: Set H = {(I — 2) {I - 1)1, {I -3)(l- 1)1, {I -3)(l- 2)1, (I -3)(l- 2) (I - 1), (I - A) (I - 1)1}. 

Step 1. For each direct descendant of H (an edge e is a direct descendant of H if e G E(H) and e is 
a direct descendant of an edge in H), check whether all its direct ancestors are in H . If so, add to H. 



\En[k- if>\ + nt - 2p> n E t \ + 

Q-(i-d+» 
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ABC 
I 

9BC 



8BC 9AC 




6BC 7 AC 89C 8AB 




4BC SAC 69C 6AB 78C 79B 89A 




3BC 4AC 59C 5AB 68C 69B 78B 79A 




2BC 3AC 49C 4AB 58C 59B 67C 68B 69A 78A 

1BC 2AC 39C 3AB 48C 49B 57C 58B 59A 67B 68A 789 
I I I I I I I I I I I I 

i i i i i i i ii ill 

Figure 2: Hessian diagram of triples on 12 vertices 

Then record the new H and record the size of the new H . Take all distinct new H with size increased by 
1 and repeat this process until there are I — 2 triples in H . Output all distinct H with I — 2 triples. 
Step 2. Performing G — [l]^ — H and output G. 

If I = 6, then G = [6]^ — H is the only left-compressed 3-graph on [6] with (g) + (t,) edges without 
containing a clique of order 5, where H = {456, 356, 346, 345}. 
It is obvious that Algorithm 13.51 leads to the following result. 

Lemma 3.6 Algorithm \ 3.5\ vroduces all possible left compressed 3-graphs on I > 7 vertices with m = 
C 3 1 ) + C 2 2 ) ed 9 es - 

As a demonstration, we verify Conjecture 11.41 when / is small in Proposition 13.71 

Proposition 3.7 Let G be a 3-graph on I vertices with m edges and contain no clique of order I — 1, 
where m = ( l g 1 ) + ( l T, 2 ) . Then A(G) < X([l - lp) for 6 < I < 12. 

The detailed proof of the case 1 — 12 will be given in the next section. 

4 Proof of Proposition 13.7 

We may verify the proposition by case analysis for each given I. Here we just demonstrate the case 
when I = 12. Using Algorithm 13.51 we list all left compressed 3-graphs on vertex set {1,2, ••■,12} with 
m = 220 = ( 12 3 ^ 1 ) + ( 12 ^T 2 ) edges without containing a clique of order 11. Let us write A = 10, B = 11, 
and C — 12 in Figure 2. 

Now applying Algorithm E31 start from initialization: let H = {ABC, 9BC, 9 AC, 9AB, 8BC}. In 
step 1, the direct descendants of H are 7BC, 8AC, and 8AB. Note that all ancestors of 7BC and all 
ancestors of 8AC are in H . We could add 7BC to H or add 8 AC to H . So there are two new H with size 
increased by 1: H = {ABC, 9BC, 9 AC, 9AB, 8BC, 7BC} or H = {ABC, 9BC, 9 AC, 9AB, 8BC, 8 AC}. 
For each of them, repeat the procedure. Continue this process until there are 10 edges in the updated 
H. Output all distinct H with 10 edges: 

• H 1 = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 89C, 7BC, 7 AC, 6BC} , 
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• H 2 = {ABC, 9BC, 9AC, 9AB, 8BC, 8AC, 8AB, 7BC, 7 AC, 6BC}, 

• H 3 = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 8AB, 89C, 7BC, 6BC}, 

• Hi = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 8AB, 89C, 7BC, 7 AC}, 

• H 5 = {ABC, 9BC, 9AC, 9AB, 8BC, 7BC, 6BC, 5BC, ABC, SBC}, 

• H 6 = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 7BC, SBC, 5BC, ABC}, 

• H 7 = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 7BC, 7 AC, 6BC, bBC}, 

• H s = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 89C, 7BC, 6BC, 5BC}, 

• H 9 = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 8AB, 7BC, 6BC, 5BC}, 

• H w = {ABC, 9BC, 9AC, 9AB, 8BC, 8AC, 8AB, 89C, 89B, 89A}, 

• Hu = {ABC, 9BC, 9 AC, 9AB, 8BC, 8AC, 8AB, 7BC, 7 AC, 7AB}. 

Correspondingly, by Step 2, we may list 11 left compressed 3-graphs on the vertex set [I] where I = 12 
with 220 edges. 

Now we may verify Conjecture [L4] by direct calculation of the corresponding Lagrangian values from 
these eleven 3-graphs. We divide our case analysis into subcases. 

Subcase 1. The set H t is removed from [12]( 3 ). In this case, E = ([H] (3) \ {9AB}) U {Cij,ij £ 
[10]( 2 ' \ {9 A, 8A, 7 A, 89}} U {1BC, 2BC, SBC, ABC, 5BC}. Observe that E A] = fori < i < j < 5. By 

©dcf T . dcf , dcf dcf , dcf dcf r dcf dcf , 

, xi — x 2 — x 3 — x 4 — x 5 — a. Let x 6 — b, x-j = c, x s — a, x 9 = e, x 10 = J, x\\ = g, x 12 = h. 

Note that 5a + b + c + d + e + f + g + h = l. In viewing of edges in E, 



A(G) = (g)« 3 + (^ja 2 (b + c + d + e + f + g) 

+5a(bc + bd + be + bf + bg + cd + ce + cf + eg + de + df + dg + ef + eg + fg) 
+(bcd + bee + bef + beg) + (bde + bdf + bdg) 
+{bef + beg + bfg) + (cde + cdf + edg) 
+{cef + ceg + cfg) + (def + deg + dfg) 

= Gi{a,b,c,d,e, f,g,h). 

Using the software Matlab, testing result shows that the maximum value of Gi(a, b, c, d, e, f, g) under 
the constraint 

5a + b + c + d + e + f + g + h=l,a>b>c>d>e>f>g>h>0. 

is < 0.1211650204992799 < A([ll] 3 ) = ^ = 0.1240. 

For the rest of 10 subcases, we only summarize our computation results as follows. 

Subcase 2. The set H 2 is removed from [12p). In this case, E = ([11] (3) \ {9AB,8AB}) U {Cij,ij £ 
[10]( 2 > \ {9 A, 8A, 7 A}} U {1BC, 2BC, SBC, ABC, 5BC}. Using the software Matlab, testing result shows 
that A(G) < 12156108878284182 < A([ll] 3 ) = ^ = 0.1240 in this subcase. 

Subcase 3. The set H 3 is removed from [12p). In this case, E = ([11] (3) \ {9AB, 8AB}) U 
{Cij, ij e [10p) \ {9A, 8A, 89}} U {IBC, 2BC, SBC, ABC, 5BC}. Matlab testing result shows that A(G) 
is < 0.1230233663938971 < ^ = 0.1240 in this subcase. 
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Subcase 4. The set H A is removed from [12]( 3 ). In this case, E = ([11] (3) \ {9AB, 8AB}) U {Cij,ij E 
[lOp) \ {9A, 8A, 7 A, 89}} U {ISC, 2BC, 3SC, ABC, 5BC, 6BC}. Matlab testing result shows that A(G) 
is < 0.12181063749855171 < = 0.1240 in this subcase. 

Subcase 5. The set H 5 is removed from [12p>. In this case, E = ([ll]^\{9AB})U{Cij,ij e [10] (2) \ 
{9A}} U {ISC, 2SC}. Matlab testing result shows that A(G) is < 0.12240632785698001 < ^ = 0.1240 
in this subcase. 

Subcase 6. The set H 6 is removed from [12] ( 3 ). In this case, E = ([11] (3) \ {9AB}) U {Cij,ij e 
[10]< 2 > \ {9A,8A}} U {IBC,2BC,WC}. Computation shows that A(G) is < 0.12187845719377105 < 
^ = 0.1240. 

Subcase 7. The set H 7 is removed from [12]^. In this case, E = ([11] (3) \ {9,45}) U {Cij,ij e 
[10] W \ {9A,8A,7A}} U {1BC, 2BC, 3BC, ABC}. Matlab computation testing shows that A(G) is < 
0.12320953232482997 < ^ = 0.1240. 

Subcase 8. The set H a is removed from [12] ^ . In this case, E = ([11] (3) \ {9AB}) U {Cij,ij e 
[10] W \ {9A,8A,89}} U {IBC,2BC,3BC,4BC}. Matlab computation testing shows that A(G) is < 
0.12205596090717179 < ^ = 0.1240. 

Subcase 9. The set H g is removed from [12p). In this case, E = ([11] (3) \ {9AB, 8AB}) U 
{Cij, ij G [10p' \ {9A, 8A}} U {1BC, 2BC, SBC, ABC}. Matlab computation testing shows that A(G) is 
< 0.12141031243761445 < ^ = 0.1240. 

Subcase 10. The set iJi is removed from [12]( 3 >. In this case, E = ([11] ^ 3 ^ \ {9AB, 8AB, 89B, 89 A}) U 
{Cij, ij e [10p> \ {9A, 8A, 89}} U {ISC, 2SC, 3SC, 4SC, 5SC, 6SC, 7SC}. Testing result shows that 
A(G) is < 0.12140780421728305 < ^ = 0.1240. 

Subcase 11. The set H n is removed from [12p). In this case, E = ([11] (3) \ {9AB, 8AB, 7AB}) U 
{12ij,ij e [10] (2) \{9A,8A,7A}}U{IBC,2BC,3BC,ABC,5BC,6BC}. Testing result shows that A(G) 
is < 0.12148307198540859 < ^ = 0.1240. 

Now the verification of Proposition 13 . 71 is completed. ■ 

References 

[1] I. M. Bomze, Evolution towards the maximum clique. Journal of Global Optimization, 10(2):143-164, 
1997. 

[2] M. Budinich, Exact bounds on the order of the maximum clique of a graph. Discrete Applied 
Mathematics, 127:535-543, 2003. 

[3] S. Busygin, A new trust region technique for the maximum weight clique problem. Discrete Applied 
Mathematics, 304(4) :2080-2096, 2006. 

[4] P. Frankl and Z. Fiiredi, Extremal problems whose solutions are the blow-ups of the small witt- 
designs. Journal of Combinatorial Theory (A), 52:129-147, 1989. 

[5] P. Frankl and V. Rodl, Hypergraphs do not jump. Combinatorica, 4:149-159, 1984. 

[6] L. E. Gibbons, D. W. Hearn, P. M. Pardalos, and M. V. Ramana, Continuous characterizations of 
the maximum clique problem. Mathematics of Operations Research, 22: 754-768, 1997. 

[7] G. He, Y. Peng, and C. Zhao, Finding lagrangians of hypergraphs. Ars Combinatoria (accepted). 

[8] T. S. Motzkin and E. G. Straus, Maxima for graphs and a new proof of a theorem of Turan. 
Canadian Journal of Mathematics, 17:533-540, 1965. 

[9] M. Pavan and M. Pelillo, Generalizing the Motzkin-Straus theorem to edge-weighted graphs, with 
applications to image segmentation. Lecture Notes in Computer Science, 2683:485-500, 2003. 



10 



[10] P. M. Pardalos and A.T. Phillips, A global optimization approach for solving the maximum clique 
problem. International Journal of Computer Mathematics, 33:209-216, 1990. 

11] Y. Peng, Q. S. Tang, and C. Zhao, On lagrangians of r- uniform hypcrgraphs, submitted. 

121 Y. Peng and C. Zhao, A Motzkin- Straus type result for 3-uniform hypergraphs, Graphs and Com- 
binatorcs, accepted, online: DOI 10.1007/s00373-012-1135-5. 

131 Y. Peng and C. Zhao, On Lagrangians of hypergraphs and cliques. In Recent Advances in Computer 
Science and Information Engineering, volume 125, pages 7-12, 2012. 

141 S. R. Bulo and M. Pelillo, A continuous characterization of maximal cliques in k-uniform hyper- 
graphs. In Learning and Intelligent Optimization, volume 5313, pages 220-233, 2008. 

151 S. R. Bulo and M. Pelillo, A generalization of the Motzkin-Straus theorem to hypergraphs. Opti- 
mization Letters, 3(2): 287-295, 2009. 

161 S.R. Bulo, A. Torsello, and M. Pelillo, A continuous-based approach for partial clique enumeration. 
Graph-Based Representations in Pattern Recognition, 4538:61-70, 2007. 

17] V. Sos and E. G. Straus, Extremal of functions on graphs with applications to graphs and hyper- 
graphs. Journal of Combinatorial Theory, Series B, 63:189-207, 1982 . 

18] J. Talbot, Lagrangians of hypergraphs, Combinatorics, Probability & Computing, 11:199-216, 2002. 

191 Q- S. Tang, Y. Peng, X. D. Zhang, and C. Zhao, Some results on lagrangians of hypergraphs, 
submitted. 

201 Q- S. Tang, Y. Peng, X. D. Zhang, and C. Zhao, On Frankl and Furedi's conjecture for 3-uniform 
hypergraphs, submitted. 

211 Q- S. Tang, Y. Peng, X. D. Zhang, and C. Zhao, On Lagrangians of hypergraphs with a specified 
subgraph, in preparation. 

221 P- Turan, On an extremal problem in graph theory(in Hungarian), Mat. Fiz. Lapok., 48:436-452, 
1941. 



11 



Appendix: Proof of Lemma 

Let m and I be positive integers satisfying ('g 1 ) < m < ('g 1 ) + ( 2 )■ Let G be a left-compressed 
3-graph on the vertex set [k] with m edges and without containing a clique of order I — 1 such that 
A(G) = Aj? m ; ^ . Let x be an optimal weighting for G with positive weights. We show that Conjecture 
[PI holds or 

|[/c-l] (3) \£|</c-2. 

Since G is left-compressed, l)k £ E. Let 6 = max{i : i(k — l)k S -E}. Since -E is left-compressed, 
then Ei = {1, . . . , i — 1, i+ 1, . . . , fc}( 2 ), for 1 < i < 6, and J5j\j = for 1 < i < j < b. Hence, by Remark 
12.4( a). we have x\ = x% = ■ ■ ■ = Xb. 

We define a new legal weighting y for G as follows. Let yi = Xi for i ^ k — 1, fe, j/fe-i = £fc-i + Xfc 
and y/c = 0. 

By Lemma l2~TT a). X(E k -i,x) — X(E k ,x), so 

HG ) y)-X(G,x) = x k (\(E k -i,x) - x k X(E k{k _ 1} ,x)) 

-x k (X(E k ,x) - ifc_iA(-Bfc(fe_i),s)) - x k -ix k X(E k( ^ k _^,x) 

6 

= x k (X(E k ^i,x) - X(E k ,x)) - xl}/Xj 

i=l 

= -bxix\. (4) 

Since y k — we may remove all edges containing k from E to form a new 3-graph G = (Jk],E) with 
X(G,y) = X(G,y) and \E\ = \E\ — \E k \. We will show that if Lemma T3.3I fails to hold then there exists 
a set of edges F' C [k — l]' 3 ' \ E satisfying 

X(F',y)>bx 1 x 2 k (5) 

and 

\F'\ < \E k \. (6) 
Then, using g]), ©, and ©, the 3-graph G" = ([k],E"), where E" = E<JF', satisfies \E"\ < \E\ and 

X(G",y) = X(G,y)+X(F',y) 
> X(G, y) + bx\x\ 
= X(G,x). 

Hence A(G") > A(G). If G" does not contains a clique of size I — 1, then it contradicts to A(G) = A 3 m l _ 1 y 

If G" contains a clique of size I — 1, then by Theorem 11.121 X(G") = X([l — 1}^) and consequently 
A(G) < X([l — 1]^) which confirms Coniecture ll.4l 

We must now construct the set of edges F' satisfying ([5]) and ©. Applying Remark l2.4f a') by taking 
i = l,j — k — 1, we have 

XiE^k^^x) 

XI = X k -l + — — — . 

X{Ei( k -i),x) 

Let C — [k — 2]( 2 ) \ E k -i- Then A(£'i\(fc_i), x) < x k Ei=b+i x i + ^(Cj Applying this and multiplying 
bx\ to the above equation (note that A(£ , 1 (j,_ 1 ), x) = Ei=2 i^k-i x i)i we nave 

. 2^, 2 , ^Etw 1 - , ^A(G,x) 
0Xia: fc < bx k -ix k -I ^ 1 r K 



i=2,i^k-l X i 2-ii=2,i^k-l X 
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Since x\ > X2 > ■ ■ ■ > x k , then 



> 2., 2n , k-jb + 2) , bx k \(C,x) 

b Xl x k < bx k - lXk (l + — — - — ) + — — - — . (7) 

Define a = [|^] and = \b(l + k ~£^ )]. Notice that [6(1 + fc ' fc ( ^+ 2) )] < k - 2 since b < k - 2. So 
P < k — 2. Let the set Fi C [k — l]' 3 ) \ F consist of the a heaviest edges in [k — 1]( 3 ) \ F containing the 
vertex k — 1 (note that \[k — 2]^ \ E k -i\ = \C\ > a). Recalling that y k -\ = Xk-i + x k we have 

UF ^ bx k X(C,x) 2 
A(Fi,y) > — — - — +ax k -!X k . 

So using 

A(Fi,y) -6xx4 > afc-xajj^a-jS). (8) 

We now distinguish two cases. 
Case 1. a > (3. 

In this case \(Fi,y) — bx k -ix\ > so defining F' = F\ satisfies ([3]). We need to check that |F'| < \E k \. 
Since E is left compressed, then [b]^ U {1, . . . , 6} X {6 + 1, . . . , k — 1} C E k . Hence 

M > Szll^zizjM > ffi_^> (9) 

since b < k - 2. Recall that |F'| = a = [^]. Since C C [k - 2} (2 \ we have |C| < ( k ~ 2 ). So using © 
we obtain 

w < r*^i < ^ < w 

So both (O and ([5]) are satisfied. 
Case 2. a < /3. 

Suppose that Lemma 1331 fails to hold. So \[k - 1]( 3 ) \E\> k-l> P + l (recall that P < k - 2). 
Let F 2 consist of any /3 + 1 — a edges in [fc - lp) \(£UFi) and define F' = F x U F 2 . Then since 
A(F 2 ,y) > ( i a + 1 - a)x|_i and using ©, 

A(F',$ -bx k -ix 2 k = \{Fi,y)-bx k - X xl + \{F 2 ,y) > (P + 1 - ~ - «) > °- 

So lU is satisfied. What remains is to check that |F' < \E k \. In fact, 

|F'|=/3 + l<fc-l<^^<|F fe | 
when b > 2. If 6 = 1, then applying (|9]), 

| P |^ + 1 = 3 < fc -2=fci±f^ll<| El | 
since k > I > 5. ■ 



13 



